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~ The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 17 October 2007 . 
2a)D This action is FINAL. 2b)[3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) IEI Claim(s) 1,2 and 7-19 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 13 Claim(s) 1.2 and 7-19 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 24 September 2003 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
aM All b)D Some * c)D None of: 

1 Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) Q Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) p aP© r No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 > □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 
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Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the 
fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. 
Since this application is eligible for continued examination under 37 CFR 1.114, 
and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the 
previous Office action has been withdrawn pursuant to 37 CFR 1 .1 14. 
Applicant's submission filed on 10/17/07 has been entered. 

Response to Amendment/Arguments 

2. Regarding claim 1 Applicant argued in substance that the brightness 
difference corrected by Matsui is caused by a difference in the exposure control 
units and does not correspond to the parallax of both eyes (P. 8, 2 nd paragraph) 

However, this argument is not persuasive because, regardless whether the 
brightness difference corrected by Matsui is caused by a difference in the 
exposure control units or not (Matsui has not expressly disclosed as such, see 
paragraph 121 recited by applicant), the difference reduced in claim 1 is a 
difference NOT corresponding to the parallax of both eyes (as a geometric 
difference would be). 
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3. Applicant's arguments in lines 4-12 (regarding the amended claim 1) and 
the last paragraph (regarding claim 19) of page 9 are moot because of new 
ground of rejections, see below. 

Double Patenting 

4. Applicant is advised that should claim 13be found allowable, claim 17 will 
be objected to under 37 CFR 1 .75 as being a substantial duplicate thereof. 
When two claims in an application are duplicates or else are so close in content 
that they both cover the same thing, despite a slight difference in wording, it is 
proper after allowing one claim to object to the other as being a substantial 
duplicate of the allowed claim. See MPEP § 706.03(k). 



Claim Rejections - 35 USC § 102 



5. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or- 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
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the treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6. Claim 1 , 2, 7-9 and 1 1 -1 9 are rejected under 35 U.S.C. 1 02(e) as being 
anticipated by Matsui et al. (US 2003/0128273). 

7. Regarding claim 1, and similarly claims 13, 15 and 17, Matsui discloses an 
image processing apparatus [Fig. 16, ref. 1-8; P. 6, paragraph 123] comprising a 
processing unit [Fig. 16, ref. 1-9; P. 6, paragraph 123] which, in a pair of images 
formed to generate a difference corresponding to a parallax of both eyes [Fig. 
17A, refs. 2-4 & 2-5; Fig. 17B, refs. 2-7 and 2-8; P. 5, paragraph 115, lines 5-6 
(indicating that the images are for the left and the right eyes and therefore are 
according to the parallax of both eyes); note that because the images are 
captured at different location, the existence of geometric difference between 
them is inherent], performs a process of reducing a difference of at least one of 
the pair of images other than a geometric difference between image structures 
corresponding to the parallax of both eyes [Fig. 17B, refs. 2-7 through 2-10 and 
Fig. 18, ref. S304 (lightness correction); P. 7, paragraph 139, especially lines 15- 
23; note that by making the intensity of corresponding portions similar, the 
intensity difference, which is not geometric, is reduced]. (Note that regarding 
claim 15, per the agreed-upon interpretation recited in the interview summary 
mailed 07/19/07 and affirmed in applicant's statement of substance of interview 
filed as part of this RCE submission, as well as paragraph 7.A of the office action 
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mailed 07/19/07, the lightness difference between the image pair that Matsui's 
invention corrects (see the analysis above for claim 1) is considered a difference 
between image structures that is not geometric.) 

Additionally, regarding the difference being super-imposed noise component, 
note that sensors (e.g., cameras) typically introduce random noise into the 
captured image frames [see, for example, Col. 1, lines 25-31 of Rashkovskiy et 
al. (US 6,563,536), which is recited here to show this well-known fact and is not 
relied upon for the rejection]. Because of its random nature, the noise (e.g., from 
dark current) contributes to the difference between any stereo image pair (since 
they will have random effects on the two images; for example, the intensity or 
color value of a pixel in one of the images may be increased due to noise but is 
decreased by noise in its corresponding pixel in the other image of the pair). 
Since Matsui corrects the difference between image pairs, it inherently corrects 
the difference between superimposed noise (such as the random noise from dark 
current) as well, since, as discussed above, the superimposed noise contributes 
to the overall difference. 

8. Regarding claim 2, Matsui further discloses that the pair of images are still 
images from a pair of video images [Fig. 16, refs. 1-1 & 1-4 and Fig. 17A, refs. 2- 
1 & 2-2 (capturing stereo video images); Fig. 17A, refs. 2-3 & 2-4 (still stereo 
image pair); P. 6, paragraphs 118, 127 and 128] formed to generate a difference 
corresponding to a parallax of both eyes [P. 5, paragraph 115, lines 5-6 
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(indicating that the images are for the left and the right eyes and therefore are 
according to the parallax of both eyes); note that a stereo pair of images 
inherently has a difference corresponding to a parallax]. 



9. Regarding claim 7, Matsui further discloses 

• a recognition unit which recognizes the geometric difference between image structures 
corresponding to the parallax of both eyes in the pair of images 

[Fig. 16, ref. 1-9; Fig. 18, ref. S303 (the largest cross correlation indicates the geometric 
difference). Note that per P. 6, paragraph 123, the CPU 1-9 executes applications and 
since it is the only unit capable of doing so (among the components of the apparatus 1- 
8), the application as the one specified in S303 of Fig. 18 necessarily has to be executed 
by the CPU and therefore it also serves as the recognition unit] 

• wherein the processing unit performs a process of reducing a difference other than 
the geometric difference between the image structures recognized by the recognition 
unit in the pair of images 

[Per the analysis of claim 1; see especially Fig. 18, refs. S303 and S304] 



1 0. Regarding claim 8, Matsui further discloses that the geometric difference 
is recognized by performing matching [Fig. 18, ref. S303, note that cross 
correlation is a form of matching]. 



1 1 . Regarding claim 9, Matsui further discloses that the processing unit 

performs, as the process of reducing the difference other than the geometric 

difference between the image structures 

• at least one of a process of removing a noise component superposed on only one of the 
pair of images from the one image or a process of correcting at least one of the pair of 
images to eliminate or reduce a difference between noise components which are different 
from each other and superposed on corresponding regions on the pair of images 
[Matsui: Fig. 18, ref. S304; P. 7, paragraph 139, lines 18-23 (correction applied to one or 
both images)] 
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12. Regarding claim 11, and similarly claims 16 and 18, Matsui further 
discloses applying correction to one or both of the pair of images [P. 7, 
paragraph 139, especially lines 21-23] and that the correction amount is 
averaged noise components [P. 7, paragraphs 142 & 143, especially the last 6 
lines of paragraph. 143; note that e(x,y) is the difference between corresponding 
pixels of the two images and (as discussed in claim 1) includes noise component 
and its average is therefore considered the averaged noise component]. 

1 3. Regarding claim 1 2, Matsui further discloses obtaining the pair of images 
of the same scene by using (1) more than one or (2) only one image pickup 
device [Fig. 16, refs. 1-1 & 1-4; Fig. 17A, refs. 2-1 & 2-2 (using two devices); P. 
5, paragraph 115, lines 5-6 (indicating that the images are for the left and the 
right eyes and therefore are according to the parallax of both eyes)] 

14. Claim 14, which is the corresponding medium claim of claims 1 
(apparatus) and 13 (method), is rejected because per the analysis of claims 1 
and 13 Matsui discloses the difference reduction process recited in the claim 14 
and further discloses a storage medium for storing applications [Fig. 16, ref. 1-10 
and P. 6, paragraph 123; see also P. 11, claim 32]. 

1 5. Regarding claim 1 9, Matsui further discloses that the non-geometric 
difference is a color difference [P. 8, paragraph 157]. 
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Claim Rejections - 35 USC § 103 

16. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

17. Claim 10 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Matsui et al. (US 2003/0128273) as applied to claims 1, 2, 7-9 and 11-19 above, 
and further in view of Aucsmith et al. (US 6,873,723) and Rashkovskiy et al. (US 
6,563,536). 

18. Regarding claim 10, Matsui discloses all limitations of its parent, claim 7. 
In particular, regarding determining noise component, note that Matsui discloses 
that to reduce a difference (e.g., lightness), the component (i.e., the correction 
amount) that causes that difference is first determined by comparing the 
corresponding regions [Fig. 18, ref. S304 and P. 7, paragraph 139, especially 
lines 15-23]. Further note that per the analysis of claim 1 the difference includes 
noise components. Matsui also discloses determining corresponding regions 
based on geometrical difference [Fig. 18, ref. S303 and P. 7, paragraph 139, 
especially lines 7-15]. 
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Additionally Aucsmith disclose that the processing unit that divides the pair of 
images into sectional regions [Fig. 5 (regions); Fig. 7, ref. 720 (dividing); Col. 5, 
line 62-Col.6, line 10; Col. 6, lines 62-64] and performs operations such as 
determining correspondence in the image pair based on the geometric difference 
on sectional region basis [Fig. 7, ref. 730 (matching, i.e., determining 
corresponding region); Col. 6, line 64-Col. 7, line 2]. 

Matsui and Aucsmith are combinable because they all have aspects that are 
from the same field of endeavor of stereoscopy. 

At the time of the invention it would have been obvious to one of ordinary skill in 
the art to modify Matsui with the teachings of Aucsmith by dividing the images 
into regions and determining regions from different images that correspond to 
each other as well as the noise component that causes non-geometric difference. 
The reason would have been to establish the correspondence between the left 
and the right images so the depth of regions can be computed [Aucsmith: Col. 4, 
lines 10-13] and that the foreground and background can be separated 
[Aucsmith: Col. 1, lines 21-35]. 

Therefore it would have been obvious to combine Aucsmith with Matsui to obtain 
the invention as specified in claim 10. 
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Conclusion and Contact Information 

19. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

• McDowell (US 6,603,535) - discloses a stereo imaging system that 
removes noise 

• Berestov et al. (US 6,714,672) - discloses a stereo imaging system that 
adjust the images for differences in brightness and/or color 

• Ishikawa et al. (US 6,549,650) - discloses compensating for luminance 
and color differences in image pairs 

20. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Yubin Hung whose telephone number is (571) 
272-7451 . The examiner can normally be reached on 7:30 - 4:00. If attempts to 
reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Matthew C. Bella can be reached on (571) 272-7778. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273- 
8300. 
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21 . Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). If you would like assistance from a USPTO Customer Service 
Representative or access to the automated information system, call 800-786- 
91 99 (IN USA OR CANADA) or 571-272-1 000. 



Yubin Hung 
Patent Examiner 
Art Unit 2624 



December 17, 2007 




